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Abstract The price of anarchy (PoA) has been widely used in static games to quantify 
the loss of efficiency due to noncooperation. Here, we extend this concept to a general 
differential games framework. In addition, we introduce the price of information (Pol) 
to characterize comparative game performances under different information structures, 
as well as the price of cooperation to capture the extent of benefit or loss a player accrues 
as a result of altruistic behavior. We further characterize PoA and Pol for a class of 
scalar linear quadratic differential games under open-loop and closed-loop feedback 
information structures. We also obtain some explicit bounds on these indices in a large 
population regime. 

Keywords Differential games • Nash equilibria ■ efficiency ■ price of anarchy • price 
of information • price of cooperation • linear-quadratic games ■ information structures 



1 Introduction 

It is well known that the non-cooperative Nash equilibrium in nonzero-sum games is 
generally inefficient 11 , which means that it would be possible for all players to do 
better in terms of attaining higher utilities or lower costs (than they would attain under 
Nash equilibria, even if the equilibrium is unique) through a cooperative behavior. This 
is true for static deterministic games, and naturally also for stochastic games as well 
as dynamic and differential games. In these latter of classes of games, one could bring 

Research supported in part by grants from AFOSR and DOE. 
Tamer Ba§ar 

Coordinated Science Laboratory and the Department of Electrical and Computer Engineering, 
University of Illinois at Urbana-Champaign, 1308 West Main Street, Urbana, IL, 61801, USA. 
Tel.: +1 217-333-3607 
Fax: +1 217-265-0997 
E-mail: basarl@illinois.edu 

Quanyan Zhu 

Coordinated Science Laboratory and the Department of Electrical and Computer Engineering, 
University of Illinois at Urbana-Champaign, 1308 West Main Street, Urbana, IL, 61801, USA. 
E-mail: zhu31@illinois.edu 



2 



up additional issues with regard to Nash equilibria beyond efficiency or lack thereof, 
such as whether an increase in information to one player (or all or a subset of the 
players) would be advantageous to that player (or groups of players), in terms of 
attaining higher utilities or lower costs, or whether acquiring more information would 
be undesirable for a player. In the special class of games where all players have the 
same utility function or cost function (that is, team problems) and what is sought is 
the global maximum or global minimum of these functions, the answer to such a query 
is clean, which is that additional information (defined as expansion of sigma fields) 
can never hurt. The same is true for the special class of zero-sum games. In stochastic 
games, or dynamic and differential games which are not team problems or zero-sum 
games, however, the answer is not that clean, and one could encounter quite surprising 
and at the outset counter-intuitive results. Perhaps the first demonstration of this 
was reported in [5] and [8], where two classes of two-player stochastic static games 
were considered, one a linear-quadratic- Gaussian (LQG) model and the other one a 
stochastic Cournot duopoly model, both of which admit unique Nash equilibria. It was 
shown that for the LQG model better information (on some stochastic variables) for 
only one player leads to lower average Nash equilibrium costs for both players, but in the 
duopoly model only the player whose information is improved benefits while the other 
one hurts (in the sense that his average Nash equilibrium cost increases) . Another way 
of comparison would be in terms of the relative values of the average Nash equilibrium 
costs attained by the players, when one player has informational advantage over the 
other. It was again shown in [5; that, in an otherwise completely symmetric game, 
the player who has better information attains higher cost than the other player in 
the LQG model (the counter-intuitive result), whereas he attains lower cost in the 
duopoly model (the intuitive result). Several manifestations of these conclusions can 
be seen also in dynamic and differential games; for example time-consistent open-loop 
Nash equilibrium is not necessarily inferior to the strongly time-consistent closed-loop 
feedback Nash equilibrium [9]. 

Now coming back to inefficiency of Nash equilibrium in a fixed nonzero-sum game, 
one question of interest is exploration of the extent of this inefficiency, that is how 
far off is a Nash equilibrium from the socially optimal solution, which is obtained as 
the maximum of the sum of the utilities of the players, or some convex combination 
of the utilities (or minimum in the case of cost functions). The notion of the price of 
anarchy (PoA)was introduced in |19] as a quantification of this offset, as a utility ratio 
between the worst possible Nash solution (among multiple Nash equilibria) and the 
social optimum. In a way, this index serves to quantify the loss of efficiency due to 
competition. It has been shown that in routing games and resource allocation games 
(see, [T5] and [IS]), PoA is bounded by a constant, allowing agents to achieve some 
level of efficiency despite being suboptimal. 

The idea of quantifying the gap between social optimality and game equilibrium 
solutions sparked many follow-up work in that same vein. In [20] , price of simplicity has 
been introduced for a pricing game in communication networks as the ratio between the 
revenue collected from a flat pricing rule and the maximum possible revenue. In [15] . 
price of uncertainty has been introduced to measure the relative payoff of an expert user 
of a security game under complete information to the one under incomplete information. 
In |22| . price of leadership has been proposed as a measure of comparison of utilities in 
a power control game between Nash equilibria and Stackelberg solutions. In all of these 
works, primarily communication networks have been used as a backdrop application 
domain, be it routing, resource allocation, power control, or security. Game-theoretical 
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methods along with Nash equilibrium have found many applications in communication 
networks, with some selected recent references being 1 3,7, 16- l8ll23| 

In this paper, we discuss several indices which quantify variations or offsets in the 
payoff values or costs attained under Nash equilibria in the context of differential games 
(DGs). We first extend the notion of PoA to DGs, which heretofore has been primarily 
limited to static continuous kernel games. We provide a characterization of PoA for a 
class of scalar linear-quadratic (LQ) DGs, and quantify the efficiency loss in the long 
run when the players behave non-cooperatively under the Nash equilibrium concept. 
We consider both open-loop (OL) and closed- loop (CL) information structures (ISs). 
We show that for the class of scalar LQ DGs with CL IS using the strongly time- 
consistent CL feedback Nash equilibrium, the PoA has some appealing computable 
upper bounds, which can further be approximated when the number of players is 
sufficiently large (that is, the large population regime), whereas, under the OL IS, it is 
possible to obtain an expression for the PoA in closed form. 

As mentioned earlier, going from static to dynamic (differential) games brings in 
the possibility of various ISs, which add richness to the (Nash equilibrium) solution of 
a game. Different ISs (generally) yield different equilibrium solutions, and hence IS is 
a crucial factor in the investigation of PoA in DGs. Motivated by this, we introduce 
another index, the price of information (Pol) , which is a result of the comparison of the 
equilibrium utilities or costs under different ISs. For the class of scalar LQ DGs above, 
we show that the Pol between the feedback and open-loop ISs is shown to be bounded 
from below by %/2/2 and from above by v2, again in the large population regime. 
Finally, motivated by some recent results reported in [3] on the level of cooperation 
between players in a routing game, captured by the degree of willingness of a player to 
place partial weight on other players' utilities in his utility function, we introduce the 
price of cooperation (PoC) as a measure of benefit or loss to a player on his base Nash 
equilibrium payoff due to cooperation. 

The structure of the paper is as follows. In Section [2] we introduce a general N- 
player DG framework with different ISs, and define in this context the indices, PoA, 
Pol, and PoC. In Section [3] we investigate the PoA for a class of scalar LQ feedback 
DGs. In Section U we study the LQ DGs under open-loop IS, and in Section [SI we 
establish bounds on the Pol. We conclude and identify future directions in Section [6] 
An earlier version of some of the results in this paper can be found in the recent 
conference paper [21] , 

2 General Problem Formulation 

In this section we first introduce the general nonzero-sum differential games framework 
along with the Nash equilibrium solution, and then introduce the three indices: prices 
of anarchy, information, and cooperation. 

Let N — {1, 2, ■ • ■ , iV} be the set of players, and [0, T)Q be the time interval of 
interest. At each time instant t G [0, T), each player, say Player i, chooses an mi- 
dimensional control value (action) ttj (t) from his set of feasible control values [/, C M. 171 ' , 
where we also make the standard assumption that as a function of t the control function 
Ui(-) is piecewise continuous on [0, T). The state variable x is of dimension n, and takes 

1 The notation ")" is introduced to capture two cases: finite horizon when T is finite (in 
which case we have [0,T]), and infinite horizon when T is infinite (in which case we have 
[0,oo)). 
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values in K n ; as a function of time, t, we assume x(-) to be piecewise continuously 
differentiable on [0,T), and evolving according to the differential equation: 

= /(«(*)> «1 (*),•' ■ ,u N (t),t) , x(0) = x , 

where xq £ IR is the initial value of the state and the system dynamics /(•) : O — > R" 
is defined on the set 

n = {(x,ui, - ■ ■ ,u N ,t)\x e R",t g [o,T),ui e Ui,i e Af} , 

as a jointly piecewise continuous function which is also Lipschitz in x, and also possibly 
Lipschitz in the Ki's, depending on whether the underlying information structure (IS) 
is open loop of closed loop feedback. 

We will consider two different ISs: Open loop (OL),, where the controls are just 
functions of time, t (and also of initial state xq, which however is assumed to be fixed 
and a known parameter of the game), and closed-loop state-feedback, where the controls 
are allowed to be functions of current value of the state and of time, that is, for Player 
i, Ui(t) = 7i(t; x(t)). In the latter case, 7$ : [0,T) x M" — > Ui is known as the policy 
variable (strategy) of Player i, which is a mapping from the set of information available 
to the player to his control (action) set0 We require each "/i(t; •) to be Lipschitz in x, 
in addition to being jointly piecewise continuous in its arguments, and denote the class 
of all such mappings by F^. We further require that / be Lipschitz not only in x but 
also in {u\, . . . ,«at}, so that the differential equation generating the state, 

= /(«(*), 71 (*;»(*))>•■ ■ ,7jV (*;«(*))»*) » ^(0) = xq , 

admits a unique piecewise continuously differentiable solution for each % £ Fi, i £ M. 
Clearly, when a particular 7^ does not depend on x (such as the OL IS), then it would 
be captured as a special case, and hence to capture this also notationally, we will write 
7t G Fi as 7^ € rV , where r\ stands for the underlying IS (which for the discussion in 
this paper is either OL or CL SF)B 

Each player i £ M is a cost-minimizer, with the objective function for Player i, as 
defined on the state and action spaces, is given by 

Li(u)= [ Fi(x(t),ui(t),--- ,u N {t),t)dt + Si(x(T)) 
Jo 

when T < 00, and 

poo 

Li{u) = / Fi(x(t),ui(t),- ■ ■ ,u N (t),t)dt 
Jo 

when T = 00, where u := {ui, . . . , kjv}- In the expressions above, for each i £ Af, the 
function F$ : Q — > R is Player i's instantaneous (running) cost function, and in the first 

2 One can introduce more general ISs, such as those that involve memory, but here we will 
restrict the discussion to only OL and CL state-feedback (SF) structures so as not to encounter 
informational non-uniqueness of Nash equilibria [9]. 

3 Even though in general different players can have different ISs, we will consider here only 
the case when the IS in the entire DG is either OL or CL SF. Otherwise, derivation of Nash 
equilibrium becomes complicated, and one has to introduce small noise robustness in order 
to eliminate informational non- uniqueness, even in LQ DCs [6], |SJ. At the conceptual level, 
however, the analysis in this paper, and the indices introduced, equally apply to the mixed IS 
case. 
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expression S% : R" — ► R is the terminal value function. Substituting Uj(t) = 7i(i; 
in the above, we arrive at the em normal or strategic form of the DG, where now 
the dependence in Lj is on 7,'s instead of Uj's. Let us denote this new cost function 
representation by Ji, for Player i, which we write more explicitly (showing its argument) 
as Ji(7' ) ), where 7^ := {7^, . . . , 7^} 6 -T'' := r± x • ■ • x _T^, where again this covers 
also the OL IS as a special case; we will occasionally drop the superscript 77 when the 
IS is clear from context. 

Let r y 1 ]_ i denote the collection of policies of all players except Player i, i.e., 7^. = 

(7i> • • •)7i_i)7jVii • • ■ >7at) ' in a S ame with IS *7- If is fixed as 7^*> Player i is 
faced with the dynamic optimization (optimal control) problem: 



In the case of infinite horizon, the problem remains the same with Si = and 
T = 00. If we denote the solution to OC(i) by 7? , and carry out the optimization for 
each i, then what we have is a Nash equilibrium compatible with the IS that defines 
the DG. This is made precise below. 

Definition 1 [n-Nash equilibrium] For a DG with IS rj, the policy N -tuple {7^ , i £ 
Af} =: -y v * is an rj—Nash equilibrium if, for each i £ TV, 7?* solves the optimal control 
problem (OC(i)). Let T^* be the set of all n— Nash equilibria, as a subset of r v . 

Now, for the CL IS case, one has to further refine the Nash equilibrium, in order 
to eliminate informational non-uniqueness. Consider a family of DGs, structured the 
same way, but defined over the time interval [s,T), where s > is the parameter that 
identifies different elements of the family. We say that an 77-Nash equilibrium, when 
r\ is the CL IS is strongly time consistent if its restriction to [s,T) is also an 77-Nash 
equilibrium, and this being true for each s and all x(s). Such Nash equilibria could 
also be called sub-game perfect equilibria, by direct analogy with a similar concept in 
finite games. We will henceforth consider only strongly time consistent Nash equilibria 
when rj is CL, but will suppress that refinement in the development below. 

Let JV* , i £ J\f, denote the achieved values of the objective functions of the players 
under a particular rj— Nash equilibrium "/' n * , and a corresponding total cost achieved 
(as a convex combination of the individual costs) be given by Jjl* = X^ieA^ Pi^i* > 
where fii is a positive weighting factor on Player i'th cost, satisfying the normalization 
condition X^gW = ^~ ^ e assume i without any loss of generality, that JV* > for 
all i G N ', and hence a fortiori Jjl > 0. 

Now as a benchmark, let us consider the case of full coordination, where the players 
agree on minimizing a single objective function, which is a convex combination of 
the individual cost functions. We may call this also a socially optimal solution. The 
corresponding underlying optimization problem is the optimal control problem: 




(1) 



s.t. ±(i) =/(a;,7i(77),7^*(r?),t), x(0) = x . 



(COC) minVfii / Ft (x(t), 7(77), t)dt + Si(x(T)) 
ier l= i [Jo 
s.t. x(t) = f(x, 7(77), t) , x(0) = x , 




4 We use "OC(i)" to denote Player i's individual optimal control problem. 

5 The acronym "COC" stands for "Centralized Optimal Control". 
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where the optimization could also be carried out with respect to control values, u, 
that is in an open-loop fashion, since the problem is deterministic and also is not 
strategic. Hence, the optimal value of this optimal control problem is independent of 
the IS, which we denote by J^, and the corresponding (open- loop) optimal control by 
u° = [u°,... , tiff]. Note that we necessarily have < ^ Jjl* , where jji* is under 
any Nash equilibrium solution out of r v * . 

Definition 2 (Price of Anarchy) Consider an N -person DG as above and its asso- 
ciated optimal control problem (COC) with J? > 0. The price of anarchy for the DG 
iE 

PN, fl ,T = 1 ™f vt , J ll*/4 ( 2 ) 

as the worst-case ratio of the total game cost to the optimum social cost. 

In addition to its dependence on the cost functions, PoA depends on the number of 
players in the game, the IS, the weights on individual players and the time horizon. 
Note that the PoA as defined in is lower-bounded by 1. 

Definition 3 (Price of Information (Pol)) Letr\\ andr\i be two ISs. Consider two 
N -person DCs which differ only in terms of their ISs, with game 1 having IS rji, and 
game 2 having 7/2 . Let the values of a particular fi convex combination of the objective 
functions be Jjl 1 * and Jjl 2 * , respectively, achieved under the Nash equilibria 7'' 1 * and 
ryVz _ 27j e p r i ce Q f information between the two ISs (under cost minimization) is given 
by 

X%(l*)= max jf I max jf. (3) 

The Pol compares the worst-case costs under two different ISs for the same convex 
combination, and quantifies the relative loss or gain when the DG is played under a 
different IS. Clearly, when XtjiCA 4 ) < 1j the IS 772 is superior to its counterpart 771 . The 
connection between Pol and PoA can be captured by Xr/i (m) ~ Pm uT I Pn u T ' 

Before introducing the third index (price of cooperation), let us define another 
class of DGs, which is an intermediate case between full cooperation and full non- 
cooperation. Consider the case where Player i, even though his cost function is Ji, 
adopts an altruistic mode and minimizes instead a cost function that places some 
weight on other players' costs. Let Aj := {A^ , j £ A/"} be a set of nonnegative parameters 
adding up to 1, J^jeA/" ~ ^' "^OVi > i G A/" be defined by 

J 4 ( 7 r ';A0 := £ ^(7"), * £ A/" 

Consider the n IS DG with cost functions J's, and let r v be the set of all its jy-Nash 
equilibria. For y v £ r v , Player i achieves an actual cost of Jii^), which may be 
better (lower) or worse (higher) than jV defined earlier. Note that if A^ = Hi for all 
i,j G A/", then all players have the same cost function, and every ry-Nash equilibrium 
solution of the altruistic game is a solution to COC, assuming that person by person 
optimal solutions of COC are globally optimal. Hence, in this limiting case we have 
full cooperation. This now brings us to the third index, which is keyed to individual 
players. 



6 If the maximum below does not exist, then it is replaced by supremum in the definition of 
PoA. 
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Definition 4 (Price of Cooperation (PoC)) Consider an N -player DG with a fixed 
IS rj, and with a fixed set of cooperation vectors X := {Aj, i £ A/"}. Let Ji, i £ M , and 
be as defined above, and P v be the set of all Nash equilibria of the original game. 
Then, the price of cooperation for Player i under the cooperation scheme X is given by 

uV(X) = max Ji(7)/ max J., (7) . (4) 

As indicated earlier, if A.; = fi for all i, where p = {m, i £ A/"} as in PoA, then 
every NE of { Ji, i £ A/"} is a person-by-person optimal solution of the COG with cost 
function J^, which would also be globally optimal under some appropriate convexity 
conditions. If 7 is one such solution, minimizing J^, then the PoC is given by 

^(M) = Ml)/ max Ji(i) , 

which can be viewed as the reciprocal of individualized PoA, where the latter is a 
measure of the loss or gain an individual player incurs on his individual cost when 
he (along with other players) plays the worst NE strategy as opposed to the globally 
minimizing strategy (again along with other players). 



3 Scalar LQ Feedback Differential Games 

The analysis of the price of anarchy is complex for general DGs as there often exist 
more than one Nash equilibrium, which show strong dependence on the underlying IS. 
For specific game structures, however, its analysis may be tractable provided that we 
avoid informational non-uniqueness. One such class is scalar linear quadratic DGs with 
state feedback IS, which is what we focus on in this section. These games also enjoy 
wide applications in economics and communication networks; see, [ID], [2]. We first 
state our model and recall some important relevant results on LQ feedback DGs; for 
details, see [9], [14]. 



3.1 Game Model 

As a special case of the class of DGs considered in the previous section, consider the 
infinite-horizon scalar iV— person LQ DGs, with quadratic cost function 

Li(u) = J (qix 2 {t) + nuj(tf)dt, ieN, (5) 

N 

x(t) = ax(t) + bjUjit), x(0) = xq , (6) 

i=l 

where q.i > 0, rj > 0, xq 7^ 0, bi 7^ are all scalar quantities. Let b := [61, ... , bjv]- We 
are interested in strongly time-consistent state- feedback (SF) Nash equilibrium (NE), 
where further the NE policies are required to be stationary (that is time invariant). We 
will refer to such equilibria in short as Feedback NE. The following theorem provides 
their characterization. 
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Theorem 1 [Feedback NE, U^, Let {ki, i G Af} solve the set of coupled algebr 
Riccati equations 




k i + q i + Si kf = 0, ieAf (7) 
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ing the stability condition a — X/»=l Si ^i < 0, where Sj := ftf/rj. Then, the 
N -tuple of policies 7* (a;) = — p-k^x, i £ A/", constitutes a feedback NE, with the corre- 
sponding cost for Player i being J* = kix\. Furthermore, the positively weighed total 
cost is = Jsxq, where k — X^fcLi t^i^i- 

If the set of coupled algebraic Riccati equations do not admit a solution which is also 
stabilizing, then the DG does not have a feedback NE. o 

The main challenge in computing the feedback NE solution for this DG is that 
equation (JT)) is a nonlinear coupled system of equations. The fact that we have a 
scalar problem alleviates the difficulty somewhat, since it is possible to turn it into a 
linear problem through a change of variables, as outlined in [12], [13] ■ Let cr^ = SiQj, 
Umax = maxj <Tj, pi = Sjfcj, i = 1, . . . , N, and 

N 



\ = J2Pi- a - ( 8 ) 



i=l 



Multiplying (0) by s.;, we rewrite it as 



pf-2\ Pi + ai=0, i = l,...,N. (9) 

Let fl C N be an index set, = Q\{i}, and uq = \Q\. For every Q 7^ 0, we have 
(after some manipulations) 

II PjX = 2n^~T | E EI Pi-E Y[pm + *Y[vA (10) 
When Q = 0, we define 

N 

JJ Pi A:= A = EPi-«- ("J 

Hence, for every Q, we have an equation in the form of either ()10[) or ()11[) . Let p = 

[l,pi,p 2 , ■ • • ,PN,PlP2, ■ ■ ■ ,PlPN,P2P3, ■ ■ ■ ,PN-lPN,-- ■ ,Y[ i= iPi] T - We can write ((TDJ 
and (|11|) into 

Mp = Ap. (12) 

Let p := [l,fci,fc 2 , . . .,k N ,kik 2 , ■ ■ ■ , k x k N , k 2 k 3 , k N _ x k N , . . . , Oi==i k i\ T and D = 
diag{l, 81, s 2 , ■ ■ ., s N , s 1 s 2 , . . ., sis N ,s 2 s 3 , sjv-lSJV, ■ • • > Ili=i s i} ■ Hence, we can 
rewrite p = Dk and (|12[) into 

Mk = Ak, where M := D X MD . (13) 

Equation (|13p is an eigenvalue problem with each index set Q corresponding to a row 
enumerated starting from the empty set. It has maximum 2 N distinct eigenvalues and 
2 N eigenvectors. The vector formed by the second entry to the N + 1-st entry of the 
eigenvectors yields the solution to ([7]l when the first entry of the vector is normalized 
to 1 and they satisfy the stability condition of Theorem [T] This leads to: 
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Theorem 2 [Feedback NE Computation, [14-1] Suppose M is a nondefective matrix 
with distinct eigenvalues. Let (A, k) be an eigenvalue-eigenvector pair such that A £ K+ 
and A > <r ma x- Then, a feedback NE y* (x) — — j?-ki x, i £ M , is yielded by k* = l T k 
provided that the resulting solution is stabilizing, where 1 = [0, 1, . . . , 1, 0, . . . , 0] T is a 
vector whose 2nd to N + 1-st entries are 1 's. 

Theorem 3 [Uniqueness of Feedback NE] Let p := YljeN'PjiP—i -~ SjeJV j^iPj ' 
There exists a unique feedback NE for the LQ DG described by {3P and under 
either one of the following two conditions: 
(i) N is sufficiently large such that p^i > a, Vi, or (ii) a — 0. 

Furthermore, the solutions to the coupled algebraic Riccati equations that characterize 
the feedback NE are of the following forms under the corresponding conditions above: 



(s-i) Pi = (p - a) - \/(p- a) 2 -di ; 

re 



(s-ii) pi = p — \J p 2 — a i, where 

/ N \ 

(14) 



" = V ■ 

Moreover, the stability condition a — YliLi s i^i < is satisfied, and hence the FB NE 
is stabilizing. 

Proof From @, we obtain 



Pi + 2(p-j - a)pi - cri = 0, (15) 



which admits the solutions: 



Pi = (a-P-i) ± V (a-p-i) 2 + Ci- (16) 

Since we need pi > 0, we retain the one with " + " sign. By rearranging the positive 
solution of (|16|l . we arrive at 

{p-af = (P-i -af +cr it (17) 

and, therefore, in terms of p, we have 

Pi = (p- a) ± \/ {p-a) 2 - o-i. (18) 

Under condition (i), we have pi — p + a < 0, hence we obtain the unique solution (s-i). 
Under scenario (ii), |T5} reduces to p, = p± \J p 2 — a, L . Since, pi < p, we again obtain 
the unique solution (s-ii). 

By summing over (JTHJ , we have a fixed point equation (|14p . Let 

Hp) ~j^(T,V(P-°) 2 -°i + <^ -(P-a). 

Its derivative is given by 

- / JV 

dP p — a 



N — 1 
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Since Oi ^ and p — a > 0, it follows that 



dp N-l\(p-a) 
1 



TV- 1 



> 0, for N > 2. (20) 



This says that P is a monotonically increasing function, and hence the solution to 
P — is unique. Hence, under (i) or (ii), there exists a unique feedback NE. 

The fact that the solution is stabilizing follows directly from (0, where the first 
term has to be negative because the second and third terms are positive. 



3.2 Team Model 

When players form a team to achieve an optimal social objective, a specific total cost 
is minimized. Let = 2i=i Vilii = diag{/iiri, . . . ,^N r N}^ an< A consider 

(FOC) min / (q fl x 2 (t)+u T (tjR f _ l u(t))dt 
«(*) JO V ' 
N 

s.t. x(t) = ax(t) + biUi{i) , x(0) = 7^ . 



The solution to this optimal control problem is standard, and is given below for 
future reference (where we suppress the dependence of q and 7? on /z). 

Theorem 4 [Centralized Optimization] The optimal control problem (FOC) admits a 
unique feedback solution which is further stabilizing. The optimal policies are 



7?(«) = --H.», K := a+ ^f+ Tb , (21) 
Hiri b 



with b := X/»=i (Pi/niTi), and minimum cost is J£ — k^x^. 
The optimal control can also be expressed in open-loop form, as: 



Ui = —kn$(t, 0)x Q , 

AW 



where <P(t, 0) is the unique solution to 



( N b 2 \ 
#(*, 0) = [a - V <P(t, 0), *(0, 0) = 1. 

V i=i / 
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3.3 Price of Anarchy (PoA) 

Here, we provide a closed-form expression for the PoA in the feedback LQ DG, where 
we make the natural assumption that xq 7^ 0, as otherwise the costs are all zero. 

Theorem 5 The PoA of the LQ feedback DG described by {5p and (jSp is characterized 
by the following: 

(i) Given a weight vector p, the PoA p^ is equal to 

P^ B = max [/i T k] /fc , (22) 



where /J, = [0, p, , 0, . . . , 0] and K, is the set of all eigenvectors of the matrix M. 
N 
j= 

N 



(ii) Suppose fj,i = fj,i :— Si / J2f=i s j>* e N ■ Then, 



pl B < [g(M) + a]/J2^k, 



FB 

[qv^i -1-0 j , 

i=l 

where £>(M) is the spectral radius o/M. 



(iii^ £ei Pniax = max ieAA Mi/si- Given a weight vector p that satisfies YLiLi ~ 1> 



t/ie Po^4 is bounded by 



pl B ^/w(f?(M) + a)/fc. (23) 



Proof The proof is a direct application of the results in Theorem [T] and Theorem [4] 
PoA is the worst-case ratio of the game cost under feedback NE to the optimum social 
cost as defined in Under the feedback IS, an LQ DG has 

FB J2iLi Pi k i ( x o) 2 p T k 
p„ — max — — t = max — ; — . 

M fc* k( x °) 2 ke/C k 

This leads to statement (i). The price of anarchy under p is 

FB J2iLl Uiki Siki 

Pn — max — - — i = max — tt 

( 24 ) 



A 



The last equality is due to ([8j). Hence, by taking the largest eigenvalue, we obtain (ii). 
The equality is achieved when £>(M) is an eigenvalue in the eigenvalue-eigenvector pair 
that yields the equilibrium from Theorem [2] For an arbitrarily picked p, (1-21) yields 

EN £4^ 7 s ^-^ N 1 

i=l 8i W ^ "max 2^i=l S i ft i 

p r . — max % max ^ — i 

M fc k fc k 

- ™„ u max(^ + a) Mmax^M) + a) 

— lilcLX ^ <^ 7 . (ZD) 

A fc fc 

Using © and taking the worst case, we obtain statement (iii). Since 

max — = 



the last inequality is achieved when p — p. 
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The next corollary further characterizes the bound on PoA. 
Corollary 1 The following follow from Theorem 
(i) Given a /i and PoA is bounded above by 

pl B < (l + ^JV + ffmax-l)) S\ (26) 

where <r ma x = maxjg_/\/ cr.;, and 

N 



Tree upper-bound is independent of /i. 
(m_) If a — 0, PoA is bounded above by 

FB ^ + _ 1}> (27) 

V^/Kin 

wreere pf nin = min ie _v' Mi/ S i- 

Proof The matrices M = [wiy ] and M = [ihij ] , i, j = 1, . . . , 2 , share the same set of 
eigenvalues. From Gersgorin theorem, we can obtain 

____ r 2" 2 s | 2» 

£>(M) ^ min < max |mj 3 - 1, max | > ^ max ^ ^ \ihij | . 

( ' 3=1 J i=l J 1 3=1 

From (JTOJ and pip , the absolute row sum RS^, k = 1, . . . , 2^, can easily be evaluated 
by letting Pi = 1: 

7?S fc = [a + ^ o-i + (iV - n n )] / [2n f2 - 1], 

where fc is the row index corresponding to the set ft. When Q — 0, we let RSi — N + a. 
From (23}, 

p£ B < [^(M) + a]/(fc/Mma X ). 

The numerator is upper-bounded by (skipping some steps): 

e\/r\ i s / ( 2a + °" max ~ 1 ) n « + N o , »r i 
o(M + a < max < max ; , 2a + iV — 1 

lis:n„<iV 2n n -l 
^ max {2a + N + <j max - 1, 2a + N - 1} 

< 2a + JV + ffmax - 1. (28) 

The second inequality holds because the quantity 

(2a + (Tmax - l)rt^ + N 
2n n - 1 

increases with uq. The denominator has a lower bound: 
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The last inequality is due to max^ m/si max; fii maxj j-. Combining (|28[) and (1291) . 

we have, for a / 0, 



When a = 0, 



min 



TV 



Using this together with (|28p . we arrive at the inequality (1271) . 



The upper bound on price of anarchy in the preceding corollary provides a worst 
case of efficiency loss. 

The next result studies the large population game and its proof relies on the Taylor 
series expansion of the square-root term in (|18p . 

Theorem 6 Suppose the number of players in the LQ DG is sufficiently large so that 

(C-i) p-i > a.VieJV" , (C-ii) a < N , (C-iii) o- max < a , 
where a — °i- Then, the following quantities can be approximated as given: 



(i) Pi 



'2a 



, (ii) ^ 



b,V2& 



(tit) J* jL=(x f , (iv) J* r J=(:ro) 2 . 

V 2a v 2a 



(v) p M ~ ; , and for a = 0, p M 



ky/2a 

Proof By Taylor series expansion, (|18|) can be written as 
Pi = (p-a) 1 - , / 1 



2(p-a) 



l + O 



(p - a)' 



(30) 



where O(-) is a function such that lim^^o O(x) = 0. In a similar way, (|14p can be 
rewritten as (skipping some steps): 



p — a 



p — a 



N - 



N 



HE 



p — a 
= N - 1 

Hence, we obtain for large TV 



Na 



2(p-a)2 



(P ~ a)'' 



l + O 



+ a 



& max 

2{p-a)' 



+ a 



(31) 
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Note that p — a > due to the stability condition. Let a — as before. Let a 

solution of (|32fl be p = \J o j2 + a, i.e., 

,-._y|[ 1+0 (S=)]. (33) 

(1331) is consistent provided that <7 ma x *C <r and a <C N. Since, by Theorem [3l the 
solution is unique under (C-i), p can indeed be approximated by p ~ a + y/a/2, which 
leads to p^ ~ -S= from 13011 . Hence, (ii)-(v) follow. 



4 Open-Loop LQ Differential Games 



In this section, we go back to the DGs described by (J5]) and (J6]), but with open-loop 
information. Each player knows only the value of the initial state of the system. Since 
the cost runs from zero to infinity, we are interested in controls that yield finite costs. 
Accordingly, we restrict the controls of the players to belong to the set 

U OL {x ) = {u G £ 2 [0,oo) j J t {x ,u) < co, Vi e A/"} , 

where £2(0, 00) is the space of square-integrable functions on [0, 00). 

Theorem 7 [Open-Loop NE, fP|?, fTJjj] Consider the N-person LQ DG in 0) and {SJ), 
and assume that there exists a unique solution to the set of equations 



N 



= 2a& + q { - & I 8j£j 



(34) 



swc/i i/iat a — X/ jLi s i?j ^ 0' uihere Sj := hf/r^. Then, the game admits a unique 



open-loop Nash equilibrium for every initial state, given by 

N 

M i it) = « ex P 




S ■ 



(35) 



The optimal cost to player i using u* is J* = k*XQ, where k* is the unique solution 
to 

2 L, - s jZj^J k i + « + s *(fi ) 2 = °- ( 36 ) 

The quantities in Theorem [7] can be made more explicit as we discuss below. By 
a slight abuse of notation, let pi := s^i as in the state- feedback information case. 
Multiplying ()34|) and Q36p by Sj, we obtain = 2apj + cr, — p^p , and = 2sjfcj(a — 
p) + <Ji + pf , where p = Jji=i Pi- Hence we can solve for pi, ki, and obtain 

Pi = 0» / (p - 2a) (37) 
h = a-i+pi / (2si(p- a)). (38) 
To obtain p, we sum (|37[1 over i and arrive at the quadratic equation p = -_f 2a . Thus, 

p=\/a 2 + a + a, (39) 
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where we have retained only the positive solution of the quadratic equation for obvious 
reasons. It should be pointed out that since the relevant p is unique, we have a unique 
open-loop NE. Using (|3tJf) . we can determine the expression for £* (and thus the OL 
NE strategies of the players [55]) . as 

& = ~T== ■ (40) 

V or + a — a 

Note that these are necessarily stabilizing, that is a — X^jLi s j£j < 0, in view of (|36|) . 
Now using (|39l) and ()37[) in ()38[) . we arrive at the closed- form expression for k*: 

k* = 1 + \ (41) 

1 2(v / ? + S-fl) 2 j' 



When a = 0, k* is reduced to 



*-Ml +! 8)- <42) 



Given weighting fi, the open-loop NE yields a total cost of 

N N 



J* = ^ViJt = y^/Xjfc*(a:o) =: k*(x y 



i=l i=l 



Since the open-loop NE solution is unique, the PoA under open loop IS can thus be 
easily found to be: 

OL 

We now capture all this in the corollary below. 



P% L = k* / k„ . (43) 



Corollary 2 The OL LQ DG of Theorem^ admits a unique OL NE given by f35\ 
and j4Cty , which is also stabilizing. Furthermore, the OL PoA is given by 



5 Price of Information (Pol) 

In the previous sections, we have introduced PoA as a measure of efficiency in going 
from cooperative to noncooperative framework, and obtained expressions for it for FB 
and OL LQ DGs . Here, we study the price of information (Pol) as a measure of 
efficiency with respect to the ISs for again the LQ DG. Following Definition Pol 
between open-loop and feedback ISs is defined by 

OL r OL* i T FB* , aa s 

Xfb — max J I max J > (44) 

fc* k* 

which can also be expressed in terms of the PoAs under the two ISs: 

OL OL , FB 
Xfb = I Pn ■ 

Using Theorem f5] we can obtain a bound on Pol: 

OL > k* 

XFB ' Mfnax(e(M)+a) • 



The following theorem further characterizes the Pol in a special case. 



16 



Theorem 8 Suppose a — 0, and the number of players is large so that N satisfies ( C- 
i), (C-ii), and (C-iii). Then, the Pol is bounded from above and below by two constants: 

2/2 sC xfb < V2. (45) 



Proof Under conditions (C-i), (C-ii), and (C-iii), we have a unique feedback NE that 
can be approximated as in statement (iv) of Theorem [6] Hence, from (|39[) we obtain 

OL J° L * >/2( Eti/W 
Xfb - tfrT - tt 1 + ~ 



jFB* 2 



qa 

N 



_ V2 Ej=iWOT ) < n i 

V L ! =iML=i ,7 »/ 
where the last inequality is obtained by noting that 

TV N N 

22 > X! ^ 0-1 ' 

i=l i=l i=l 

The lower bound can be achieved by noting that <7j,gj,/ij are all nonnegative. 



Theorem[8]is useful in the design of games via access control or pricing mechanisms. 
Let x G (^r,V2] be some target Pol to achieve so that x'fb ^ X- F° r example, when 
X = 1, it means the game needs to be designed so that the open-loop NE yields no 
larger cost than the feedback NE. Hence, a necessary condition to meet such a design 
criterion is: 

E ' gA ^ g ^ ^ y/2 X % 1. (46) 
qa 

An access control is to admit a set M of players so that (|46[1 is satisfied when all 
the system and player parameters are given. When set M is fixed and not adjustable, 
we may use "pricing" mechanisms to control the parameters 7j or qi, which reflect the 
unit "price" of penalty on the control effort and the state, respectively. In the following 
corollary, we capture the special case of homogeneous players. 

Corollary 3 Suppose the LQ DG satisfies the conditions in Theorem^ In addition, 
let the players be symmetric so that o~i — a,pi — p, Vi £ N ■ When N ^ 3, the open- 
loop IS yields better total optimal cost; otherwise the FB information does better. In 
addition, as N — > oo, limjv_»oo Xfb = a ^ the rate of O (jj-)- 

Proof The proof directly follows from Theorem [8] The price of information under 
the additional assumptions becomes x'fb = "^75 (l + Tv')' ^ * s independent of the 

parameters of the players and approaches ^ as N — > 00. By letting x'fb ^ 1, we 
obtain TV ^ 1 / (s/2— 1) . Hence, since TV is an integer, the open-loop NE does better 
than the feedback NE when there are 3 or more players. 



Theorem [5] and Corollary [3] have implications in the design of games via access 
control when open loop is the preferred mode of play. 
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6 Applications and Illustrations 

In this section, we apply the results obtained heretofore to two classes of application 
scenarios in flow control. 



6.1 Multiuser Rate-Based Flow Control 

We adopt here the communication systems model described in [2], where the players are 
the users or sources, and the action (control) variables are the flows into the network. 
If a link receives more total flow than what it can accommodate (measured by its 
capacity), then packets queue up. Having long queues is not desirable, because it leads 
to delays in transmission. We call such links which are congested bottleneck links, and 
formulate the game around one such link. Let qi (t) denote the queue length at such a 
bottleneck link and let s(t) denote the total effective service rate available at that link. 
Assume that each user is assigned a fixed proportion of the available bandwidth; more 
specifically, the traffic of source i,i = 1,2, N , has an allotted bandwidth of u>is(t), 
where uij's are positive parameters which add up to 1. We assume that the users have 
perfect measurement of s(t), but occasionally exceed or fall short of the bandwidth 
allotted to them due to fluctuations. Hence, if di(t) denotes the rate of source i at 
time t, we can introduce Ui(i) := di(t) — WiS r (t) as the control (action) variable of the 
source. Then, queue build-up is governed by the differential equation 

N 

«(t) = £«i(t), ( 47 ) 

i=l 

where we assume that queue is relatively tightly controlled so that end effect constraints 
(starvation and exceeding an upper limit) do not become active. The goal is to ensure 
that the bottleneck queue size stays around some desired level qi, and good tracking 
between input and output rates is achieved. Toward that end, we consider the shifted 
variable x(t) := qi(t) — qi, which satisfies the following differential equation which is 
the shifted version of (1471) : 

JV 

x(t) = Y, u i x(0)=x . (48) 
i=i 

We now consider a noncooperative scenario in which each source determines a linear 
feedback policy (or an open-loop policy) to minimize its own individual cost function 

Hu) = J o (\x(t)\ 2 + \u t {t)\ 2 ^dt, (49) 

which is consistent with the overall goal of keeping x and u,'s small. We can also 
consider a related team problem in which sources minimize cooperatively a common 
cost under the same information structure (where as we know actually the IS does not 
make a difference in this case): 

JV 



m = U\x(t)\ 2 +j2\Mt)\ 2 )dt. (so) 
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This is now within the framework of LQ DGs studied earlier, with the correspon- 
dences being a — 0, xq — 1, <Tj = Sj = <jj = = 6j = 1 in ([5} and ((6]). To obtain some 
numerical results, let us take xq = 1. 

In the case of the 2-person LQ feedback game, the M matrix introduced earlier 
becomes 

"0 1 1 ' 

10 0-1 
10 0-1 
1/3 1/3 

and if N = 3, we have 



M 2 = 



M 3 = 



1 1 1 
10 0-1-10 

1 -1 -1 
1 -1 -1 

1/3 1/3 -1/3 
1/3 1/3 -1/3 
1/3 1/3 -1/3 
1/5 1/5 1/5 



The positive eigenvalue of M2 is A2 = 1.1547 and the corresponding vector is P2 = k2 = 
[1.0000, 0.5774, 0.5774, 0.3333] T . The sum of the optimal costs under equal weights is 
J2 = 0.5774 while the optimal common cost is Jf = 0.5, yielding the price of anarchy 
value p£f = 1.1547. For the case with 3 players, the eigenvector is found to be P3 = 
k 3 = [1.0000,0.4472,0.4472,0.4472,0.2000, 0.2000, 0.2000, 0.0894] T corresponding to 
A3 = 1.3416. Again under equal weights, the total NE cost is J3 = 0.4472 and the 
minimum social cost is Jf = 0.3333. Hence, the price of anarchy is given by = 

1.3416. When the number of players becomes large, p^ B ~ fr° m Theorem HI] 

In the case of open-loop flow control, we obtain k* = -7= (3 + 277 ) an d total NE 
cost as Jjy = k* . In the 2-user game, J 2 = 0.5303 yielding the price of information 
x'fb — 0.9184. The open-loop NE thus yields 8.16% less cost in comparison to the 
closed-loop FB one. In a 3-user game, J3 = 0.3849, leading to a price of information 
value of Xfb = 0.8607, which yields a 13.93% more cost for the FB IS case. We also 
note that as the number of players increases, the open-loop IS yields a cost approaching 
0, i.e., limTv-i-oo Jn ~ 0, while in the feedback case, even though it still converges to 
0, the rate is slower: J* ~ ®' ^ 6 °'- ,serve ^ na ^ Xfb g° es to ^ at a rate of jj- 

as JV gets large, i.e., 



lim 



iV- 



OL 

Xfb 



V2 



+ 



2N 



V2 
2 ' 



It is also noted that open-loop NEs always yield less equilibrium costs even though 
they require less information. 

Due to the symmetry of players in the flow control problem, we can obtain exact 
closed-form solutions to the equilibrium costs using (|18p and (|14p without approxima- 
tion. It is not hard to show that under equal weights, 
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Pol: Open-loop vs. Feedback Information 




'1 23456789 10 

N 



Fig. 1 Price of Information 



In Figure[T] we show the price of information under open- loop and feedback information 
structures, and in Figure [2] we show the corresponding prices of anarchy. By exact 
calculation, we find when N = 4, the open-loop NE cost to be J4 = | = 0.3125, which 
catches up with and becomes better than the feedback NE cost: J| = = 0.378. This 
is consistent with our earlier observation based on large population approximation. 

We observe in Figure [1] that the NE costs are the same at N = 1 (as they should 
be), and as N increases, both open-loop and feedback NE costs decrease. As N becomes 
large, both costs approach 0. This happens because the queue length is fixed. When 
the number of players goes to infinity, the contribution from each user is negligible. 
Moreover, the state x(t) can be driven to zero very fast as the amount of total con- 
trol effort increases with the number of players. The cost incurred from the transient 
behavior of x(t) then goes to zero. In addition, for N ^ 2, open-loop NE yields better 
costs. The price of information Xfb ^ s always below 1 but maintains its level above 
■^2 . I n Figure [2] the price of anarchy starts at 1 when N = 1 and increases as the 
number of players grows. The cost under the feedback NE grows faster than the one 
under open-loop NE. 

6.2 Normalized Flow Control Dynamics 

In this section, we investigate a general flow control dynamics, which differs from (14811 
by inclusion of a population-dependent normalization factor f(N), where /(•) is an 
increasing function of N: 

1 - 

m = W)^ Ul ' x{0) = 1 - (51) 

i—1 

The introduction of a normalization factor is to adjust the queue length proportionally 
when the number of users increases. 
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PoA: Open-loop vs. Feedback Information 




Fig. 2 Price of Anarchy 



Proposition 1 The prices of anarchy p® L , pf^ B , and the price of information x'fb 
are independent of the normalization factor f(N), as summarized in Table\]\ 



Table 1 Various indices for normalized flow control game 



J* (FB) 


J° (TP) 


J* (OL) 


Pit 




x3 L B 


/(AO 


/(AO 


/(AO (1 , 1 \ 
VAT *-2 T" 


N 




\l 2 - ~k (1 + if) 




JV 


V2AT-1 



Proof Using (|18|l and (I14|l . we obtain for a given TV as follows: 

TV 1 

p 

Pi ■ 



/(TV) T ! 
1 



/(TV)^]? - T ! 
f(N) 



s t V2iV - 1 ' 

AT 



1=1 

The team problem yields an optimal cost of 



lq _ f(N) 

b TV ' 



(52) 
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Table 2 Indices under two normalization factors using the large population approximation 



f{N) | J* (FB) | J° (TP) | J* (OL) | p FB I p° L | x % 



1 


1 

\/2JV 


1 

JV 


v^jV V 2 2iV / 


[~N 

V 2 




\/2 , 
2 ~ r 2JV 


1 

JV 


In 

V 2 


1 


^(§ + 2*) 


In 

V 2 


V^(f +2lv) 


2 2iV 



Hence, the price of anarchy p^ B under the state-feedback information structure is 
independent of f(N), and is given by 

p » =7W=i (53) 

The open-loop price of anarchy is also independent of the factor f(N). Since J* = 
+ it is given by 

The price of information is also independent of f(N), and given by 



Xfb 



As a case study, we let f(N) = Then, bj = jj- , s,; = crj = for all i 6 A/". 
When the population is large, we have J* ~ ant ^ ^* ~ (2 + • The price of 

anarchy remains p ~ y^"- The price of information remains x'fb = ^ 'TN ~* as 
TV — > 00. It can be shown that x'fb does not change with the factor f(N). In Figures[3] 
and |31 we show the prices based on the exact closed form solution obtained in the same 
fashion as in the previous section based on (|18[) and (|14[) . We observe that the open- 
loop NE always outperforms the feedback equilibrium. It should be pointed out that 
(i) in Figure the open-loop and feedback costs increase with the number of users. 
This is due to the introduction of normalization factor into the system dynamics. We 
allocate the queue length as an increasing function of the number of users; (ii) Figures 
[4] and [2] are identical due to the above proposition. 

If we set /(AO = VW, we have the open-loop and feedback optimal costs approach 
J; and respectively, as A — s> 00. Figure [5] demonstrates that result. 

A summary of the results with /(A) = 1 and /(A) = jt under large population 
approximation is provided in Tabled 



7 Conclusion 

In this paper, we have introduced the notions of price of anarchy, price of information, 
and price of cooperation for nonzero-sum differential games, have studied the first two 
extensively for a class of scalar linear-quadratic differential games, and have obtained 
bounds and approximations on them, with computable bounds available in the large 
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population regime. Future promising work is to extend these results to non-scalar 
differential games as well as to obtain their counterparts for the price of cooperation. 
Also computing these indices for specific models from communication networks and 
economics would be a fruitful area of research. 
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